Haptic force-feedback for computing interfaces

ABSTRACT

Methods, systems, and apparatuses, including computer programs encoded on computer-readable media, for receiving, from an accelerometer, angle information of a device. A calibration position is measured based upon the angle information of the device when the device is at rest. A current position is constantly measured based upon the angle information of the device. A current pressure is calculated based upon the current position and the calibration position. The current pressure can be used to augment an audio signal and/or used to augment an audio instruction.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.14/949,766 filed Nov. 23, 2015, which is a continuation-in-part ofInternational Application No. PCT/US2014/039398 filed May 23, 2014,which claims benefit of priority to U.S. Provisional Application No.61/827,390 filed May 24, 2013, all of which are incorporated herein byreference in their entirety.

BACKGROUND

Mobile computing devices (MCD) have made an enormous impact in thepublic consumer space with explosive popularity for all types of usersranging from technology experts to computer illiterate users. Examplesof MCDs include tablets, smart phones, phablets, portable music players,personal digital assistants, etc. These devices created a paradigm shiftwhere the traditional functionality of the mobile phone has transformedfrom a paradigm concerned with receiving and making calls to providing apowerful, poly-modal mobile computing device with telecommunicationcapabilities. For the majority existing MCD software applications, theprimary means of interaction with an MCD is through the touch-screen—bypressing buttons or using swipe gestures. Force-feedback, however, ismissing. For interfaces that utilize trackpads as found commonly inlaptops, some desktop computers, and touch-screen MCDs, force-feedbackis literally absent due its “touch-screen” interface designs. This isespecially the case for MCD as the primary mode of content interactionis via the multi-touch-screen display.

SUMMARY

In general, one aspect of the subject matter described in thisspecification can be embodied in methods for receiving, from anaccelerometer, angle information of a device. A calibration position ismeasured based upon the angle information of the device when the deviceis at rest. A current position is constantly measured based upon theangle information of the device. A current pressure is calculated basedupon the current position and the calibration position. In oneimplementation, the current pressure is used to augment an audio signal.In another implementation, the current pressure is used to augment anaudio instruction. Other implementations of this aspect includecorresponding systems, apparatuses, and computer-readable media.

In general, one aspect of the subject matter described in thisspecification can be embodied in a system comprising an accelerometerconfigured to provide angle information of a device. One or moreprocessors configured to receive a reference position that continuouslymeasures a current position based upon the angle information of thedevice. A current pressure determined based upon the current positionand the reference position, a first layer of resistive materialcomprising two or more removable sections attach to the deviceconfigured to compress when the device is pressed and a second layer ofresistive material attached to the first layer of resistive materialconfigured to compress when the device is pressed. The one or moreprocessors are further configured to measure a calibration positionbased upon the angle information of the device when the device is atrest with no pressure applied, wherein the calibration position is thereference position, and the accelerometer is used to measure the amountof tilt with respect to the calibration position of the device at rest,and tilt is dependent upon the pressure and movement of the firstresistive material layer and the second resistive material layer.

The foregoing summary is illustrative only and is not intended to be inany way limiting. In addition to the illustrative aspects,implementations, and features described above, further aspects,implementations, and features will become apparent by reference to thefollowing drawings and the detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other features of the present disclosure will becomemore fully apparent from the following description and appended claims,taken in conjunction with the accompanying drawings. Understanding thatthese drawings depict only several implementations in accordance withthe disclosure and are, therefore, not to be considered limiting of itsscope, the disclosure will be described with additional specificity anddetail through use of the accompanying drawings.

FIGS. 1 and 2 illustrate two layers of foam for use with a user inputdevices in accordance with an illustrative implementation.

FIGS. 3A, 3B, and 3C illustrate a force-feedback mobile computing devicesetup in accordance with an illustrative implementation.

FIG. 4 illustrates two foam layers for use with a mobile computingdevice in accordance with an illustrative implementation.

FIG. 5 illustrates another force-feedback mobile computing device setupin accordance with an illustrative implementation.

FIGS. 6A, 6B, 6C, and 6D illustrate another force-feedback mobilecomputing device setup in accordance with an illustrativeimplementation.

FIGS. 7A, 7B, and 7C illustrate a case that incorporates resistivematerial in accordance with an illustrative implementation.

FIGS. 8A, 8B, and 8C illustrate different pressure types used in aforce-feedback mobile computing device in accordance with anillustrative implementation.

FIG. 9 illustrates an elastic band providing force-feedback for a GUIelement in accordance with an illustrative implementation.

FIGS. 10A, 10B, and 10C illustrate elastic bands incorporated into acase in accordance with an illustrative implementation.

FIG. 11 illustrates an elastic band providing force-feedback for aninvisible GUI element in accordance with an illustrative implementation.

FIG. 12 illustrates an elastic band providing force-feedback inaccordance with an illustrative implementation.

FIG. 13 illustrates an elastic band providing force-feedback to flipthrough pages in accordance with an illustrative implementation.

FIG. 14 illustrates an elastic band providing force-feedback for a GUIelement in accordance with an illustrative implementation.

FIG. 15 is a block diagram of a computer system in accordance with anillustrative implementation.

FIG. 16 is an illustration of an implementation utilizing conductivestrings.

FIG. 17 is an illustration of an implementation with an alternativestring storage mechanism.

FIGS. 18A and 18B are illustrations of an implementation utilizingelectroactive polymers.

FIG. 19A illustrates a front view of one embodiment of a mounting systemattached to rearview mirror; FIG. 19B illustrates a side view of themounting system of FIG. 19A.

FIGS. 20A, 20B, 20C, 20D and 20E illustrate views of a MCD in a casehaving a mounting system; FIG. 20A shows the mounting system stowed,FIG. 20B shows the mounting system extended; FIG. 20C shows the mountingsystem engaging a rear view mirror; FIG. 20D shows the mounting systemsecured to a rear view mirror; FIG. 20E shows the mounting systemwherein the height is adjusted by partially retracting the mountingsystem within the case.

FIG. 21A illustrates a partially retracted engaged mounting systemhaving an indentation; FIG. 21B illustrates a partially retractedengaged mounting system having a second horizontal handle below therearview mirror and also having an indentation.

FIG. 22A illustrates the mounting system detaching and initializing arear view mirror; FIG. 22B illustrates the MCD of FIG. 22A with themounting system stowed.

FIG. 23 is a cut-away of the MCD in a case with the mounting systempartially stowed;

FIGS. 24A and 24B depict simplified device of FIG. 23 for illustrationpurposes; FIG. 24A shows the mounting system as it is being extended;FIG. 24B shows the mounting system extended with a lower retention baris engaged with a blocking module.

FIGS. 25A, 25B, and 25C illustrate a locking mechanism; FIG. 25A shows atop view of a portion of the locking mechanism with a roundedimplementation tooth mechanism; FIG. 25B illustrates a perspective viewof a portion of the locking mechanism with the locking mechanismpointing towards the left hand side and the track at the bottom of thesupport handle; FIG. 25C illustrates a side view of a portion of thelocking mechanism where the locking teeth point away from the page andthe track keeps the dual handle securely in place and aligned.

FIG. 26A shows a portion of the locking mechanism where the verticalsupport is straight and engaged with the lock rail; FIG. 26B shows thevertical support bent away from the lock rail allowing the verticalsupport to slide relative to the case, with the retention moduleengaging to limit the bending of the vertical support.

FIG. 27 shows another implementation of the top horizontal handleportion of mounting system 2001 where the horizontal handle residesbelow a flange portion of the vertical support.

FIGS. 28A, 28B, and 28C illustrate an embodiment having a filament asthe mounting system 2001′ FIG. 28A shows a vertically stowed filament;FIG. 28B shows a horizontally stowed filament; FIG. 28C shows ahorizontal filament engaged with a support 22.

Reference is made to the accompanying drawings throughout the followingdetailed description. In the drawings, similar symbols typicallyidentify similar components, unless context dictates otherwise. Theillustrative implementations described in the detailed description,drawings, and claims are not meant to be limiting. Other implementationsmay be utilized, and other changes may be made, without departing fromthe spirit or scope of the subject matter presented here. It will bereadily understood that the aspects of the present disclosure, asgenerally described herein, and illustrated in the figures, can bearranged, substituted, combined, and designed in a wide variety ofdifferent configurations, all of which are explicitly contemplated andmade part of this disclosure.

DETAILED DESCRIPTION

The touch-screen interfaces of computing devices generally lack animportant feature: haptic force-feedback. This friction-based feedbackplays an essential role in most human-machine interaction scenarios,whether when driving an automobile (steering, stepping on theaccelerator), typing on the keyboard, or when playing traditionalmusical instruments (guitars, pianos, percussion instruments, etc.). Theabsence of force-feedback can diminish user control and interactionexpressivity whether in the context of a gaming environment, musicalinstrument application, or when engaging with something simple as anInternet browser. This specification discusses providing force-feedbackto a user of a user input computing device utilizing resistive materialsuch as foam padding. In another described embodiment, force-feedback isprovided using elastic bands with strategic use of sensors and softwaredesigns. In addition, this specification describes how force-feedbackcan be added to existing user input computing environment, andespecially MCDs. In various implementations, the following fourphilosophies were kept in mind: (1) avoidance of permanent physicalalteration to the device itself, (2) easiness in adding force-feedbackto the device, (3) flexibility in modifying expressivity via hardwareand/or software, and (4) exploitation of polysensory features of modernuser input devices such as MCDs. Various implementations essentiallyaugment expressivity by providing force-feedback for MCDs, for example.In one implementation, force-feedback can be achieved through acombination of foam padding and exploiting on-board accelerometers foundin many standard MCDs. In other implementations, a padding material thathas physical resistive reaction properties can be used to achieveforce-feedback. In another implementation, this can be achieved throughcombination of elastic bands and utilizing standard graphical userinterface (GUI) sliders also found in most MCDs.

For the majority of the touchscreen-based interactions, force-feedbackis absent although it is critical in human-machine interactionsituations. This is especially the case when using MCDs orcomputer-based user interfaces for musical expressivity including liveperformances with electronic instruments or when interacting withphysically modeled software instruments.

Hardware Designs

Musical controllers are traditionally designed in two ways: (1) usingoff-the-shelf sensors and microcontrollers by permanently orsemi-permanently attaching to them existing musical instruments; and (2)by building new instruments from scratch. These two types of designmethodologies have the advantage of providing force-feedback but alsosuffer from issues including fragility, hardware maintenance,mass-production difficulties, accessibility, and high development cost.With the advent of modern tablets, smart-phones, “phablets,” and mobilecomputing devices in general, an additional design method can beconsidered to the way controllers are designed and built. This thirdapproach exploits the inherent polysensory capabilities and efficient,all-in-one hardware features of MCDs typically include multitude ofon-board sensors, wireless communication capabilities (WiFi andBlueTooth), computing power, battery life, and its mass appeal. ModernMCDs include a touch-screen, sophisticated accelerometer, alongside withbuttons and cameras. MCDs are furthermore robust, take on amultifunctional role in daily life of modern women, men, and children,and are used ubiquitously by the general public. The primary mode ofinteraction for MCDs is the touch-screen. One serious deficiency withtouchscreen-based MCDs as an expressive controller is the lack offorce-feedback. Various implementations described herein describe anexpressive force-feedback solution for touchscreen-based MCDs, whichallows for augmenting MCD interaction expressivity and control in amultitude of applications ranging from, but not limited to, musicalsoftware, gaming software, business software, and general Internetbrowsing software.

Force-Feedback Using Resistive Materials

In one implementation, a resistive material, such as, foam paddingmaterial can be used. Other resistive materials, such as gels, one ormore springs, plastics, etc., can be used.

FIGS. 1 and 2 illustrate two layers of foam for use with a mobilecomputing device in accordance with an illustrative implementation. Asseen in FIG. 1, there are two layers of foam. The top layer 102 is madeup of smaller, equal-sized modular foam blocks 104 a-g firmly placed inseries on a single foundational foam pad 106. The smaller and modulartop blocks 104 a-g can be kept securely affixed to the bottom foundation106 with adhesive materials such as hooks with loop fasteners 108 asshown in FIG. 2. In other implementations, the foam blocks 104 a-g arenot affixed to the foundation 106. In yet other implementations, thefoam blocks 104 a-g are affixed to the foundation 106 using adhesive. Inthe implementations using hooks with loop fasteners 108 or otherattachment methods that allow for the removing of foam blocks, quickforce-feedback adjustment and feel can be accomplished. For example,removing foam blocks can result in less resistance and smootherforce-feedback and vice-versa. FIG. 2 shows three blocks removed toallow for softer force-feedback feel—a type of “subtractive-resistance”design approach.

FIGS. 3A, 3B, and 3C illustrate a force-feedback MCD setup in accordancewith an illustrative implementation. A foam block 202 can be placed onor at the edge of a surface 204, such as a table, resulting in an angledtablet configuration. When pressing on the touchscreen, force-feedbackis accomplished by the absorption of pressure by the foam block 202during user-touchscreen interaction. The on-board accelerometer of theMCD 206 is used to measure the amount of tilt with respect to its restposition when no pressure is applied. The accelerometer measurements canthen be mapped to software-specific parameters. For example, for amusical instrument application the readings could be mapped to controlmusical parameters such as tremolo, vibrato, general aftertouch, filterbehavior, amplitude envelope, and velocity. Specific mapping strategiesallows for music augmentation that is being played or produced by theMCD 206. This same setup can be used for any other type of softwareapplication scenario including opening a door or depressing theaccelerator in computer gaming applications.

A number of foam configurations have been tested and explored to rendera robust and expressive “default” configuration. FIG. 4 illustrates onesuch “default” configuration. This configuration allows for substantialexpressive control based upon the default tilt angle. The tilt angle,when the MCD 206 was pressed, can be directly proportional to expressiverange. However, as described in greater detail below, otherconfigurations such as a greater tilt angle, a smaller tilt angle, or notilt angle are also possible depending on the desired application.

Another configuration uses multiple layers of foam (two in example), butwithout a tilt angle. FIG. 4 illustrates two foam layers in accordancewith an illustrative implementation. In this implementation, theconfiguration of the foam layers 402 and 406 allows for a moredistributed pressure configuration providing possibilities forexpressive 2-dimensional mapping. The MCD can be positioned, such thatthe MCD is substantially parallel to a surface on which the foam layers402 and 406 rest. Accordingly, there is no tilt angle between the MCDand the surface. When pressure is applied to the MCD, the MCD will tiltbased upon the pressure and movement of the foam layers 402 and 406. Theaccelerometer can be used to measure the amount of tilt using the x andy accelerometer readings. This configuration can also be applied fortransient response interactions (e.g. quick strike with force-feedback).In this scenario, the accelerometer's z-axis can be used to detectvertical acceleration-type interactions with added force-feedback. Allof the interactions scenarios allow for force-feedback interactions withMCDs

FIG. 5 illustrates another force-feedback mobile computing device setupin accordance with an illustrative implementation. Foam blocks 502 a and502 b can be placed below the MCD 206. In this configuration, the MCBcan tilt towards as well as away from a user.

FIGS. 6A, 6B, 6C, and 6D illustrate yet another force-feedback mobilecomputing device setup in accordance with an illustrativeimplementation. A foam block 602 is placed underneath the MCD 206. Thefoam block is positioned towards the middle of the MCD 206. This allowsthe MCD to tilt towards or away from a user, as the user presses on theMCD 206. FIG. 6B shows the MCD in a neutral position such that MCD isbalanced on the foam block 602. FIG. 6C shows the MCD in a tiltedposition toward the user. FIG. 6D shows the MCD in a tilted positionaway the user. This design allows for additional control andforce-feedback as tilt is possible both towards and away from the user.

In another implementation, the foam padding can be secured to the MCD.In this implementation, the foam padding is secured to the MCD usingadhesive, Velcro®, elastic bands at the top and bottom, or in the formof a custom MCD case commonly used for protecting smart-phones and MCDsin general. This setup enables the user to comfortably hold the MCD(e.g. a smaller smart-phone) in one hand while interacting with thedevice with the free hand. This interaction configuration providesforce-feedback interaction and also allows great mobility as the user isno longer tied to a surface. In effect, the one hand holding the MCDacts like a table and the other hand can be used to control the MCDwhile the setup as a whole provides force-feedback and mobility.

In one implementation, the resistive material can be integrated into acase that holds the MCD. FIGS. 7A, 7B, and 7C illustrate a case 700 thatincorporates resistive material. The case 700 holds a mobile computingdevice 702 and incorporates the resistive material 704. Theseimplementations can be helpful as they have dual functionality: (1)acting to provide force-feedback and (2) protecting the MCD when intransit, for example.

Force-Feedback Measurements

In various implementations, force-feedback measurements are based uponreadings from the accelerometer of the MCD. These readings, incombination with resistance from the foam/cushion material, can be usedto mimic force-feedback-based user interaction. In certainimplementations, a calibration of the system is done when the MCD is inits rest position. For example, in FIG. 3, the calibration is done withthe MCD resting on the foam padding 102 without pressure appliedimproving performance. In one implementation, calibration can beaccomplished by pressing a reset button in the GUI of the MCD. In someimplementations, there is also a second calibration procedure to allowfor improved expressivity. This second calibration procedure includescapturing the maximum displacement of the tablet (at rest to fullydepressed) in order to set the maximum accelerometer range for a givensession. This second calibration procedure, however, can be bypassed anduse the default fixed movement range instead.

Pressure Sensitivity Options

In lieu of physically changing the sensitivity of the force-feedback, anumber of software pressure settings can be used. These can be readilyaccessible via a typical GUI setup/configuration interface. FIGS. 8A-8Cshow various types of pressure settings in accordance with anillustrative implementation. These pressure settings can be used to mapaccelerometer data to a pressure amount. As described above, thepressure amount can be used to augment user interaction-basedexpressivity and control. The accelerometer data and pressure values canbe normalized. For example in these examples, the accelerometer data isnormalized to values between 0.0 and 1.0, with 0.0 representing a tiltamount at rest and 1:0 representing a maximum tilt or movement amount.The pressure values can also be normalized to values between 0.0 and1.0, with 0.0 denoting zero pressure and 1.0 the maximum pressure thatthe system is able to track. The pressure can then be used to augmentuser expressivity and control. For example, the pressure amount can bemapped to the amplitude or the pitch of a software-based musicalinstrument. The mapping can alter the audio signal in various ways. Inanother example, the minimum amount of pressure could leave theamplitude/pitch unchanged, where a maximum pressure amount could producedouble the amplitude/pitch of the software instrument and linearlychanging values between the minimum and maximum ranges. This could alsobe extended to other functions like scroll speed for GUI interfaces,velocity characteristics of objects in computer games, pressuresensitive button, sliders, and pan-pots, etc.

FIGS. 8A-8C illustrates different pressure types that can be used in aforce-feedback mobile computing device in accordance with anillustrative implementation. This includes the linear mode, exponentialmodel, and a logarithmic mode. The logarithmic mode can be efficientlyimplemented with a single equation. Additionally, a custom mode can alsobe defined by the user via a GUI input, for example (8C).

The straight line in both FIGS. 8A and 8B is the simplest mappingfunction with a 1:1 input to output relationship, where the input andoutput follow the general relationship expressed as y=x^(p). Here, y isthe output and x the input accelerometer readings. In the linear mode,the power parameter p is a constant: p=1. This setting does not bias theinput and output relationship and can be particularly expressive inrendering tremolo and vibrato performance techniques. Vibrato andtremolo-like interaction can be simply achieved by rubbing the GUIbutton which is similar to guitar playing techniques and akin to“channel aftertouch” that is provided for high-end electronic musicalkeyboards.

The exponential mode shown in FIG. 8A whereby the power parameter p isset to positive value greater than 1.0. The greater the p, the morecurvature in mapping the input and output. The interaction result isthat more pressure will have to be exerted for affecting the output thegreater the value of p. The greater the value of p, the closer theinteraction will be to a “button” (on/off) rather than a sliding-typeinteraction interface (continuous).

The “logarithmic” pressure interaction is shown in FIG. 8B which can beapproximated by using a power p value that is bounded by limiting it to:0<p<1. As before, when p=1, the mapping is linear. If p>1, the behavioris like an exponential function. If p is between 0<p<1, then theinteraction will be an inverse exponential mapping of input to output.FIG. 8B shows the mapping output starting p=1 (linear) and ending atp=0.1 at 0.1 decrements. The logarithmic pressure mapping allows forhighly sensitive interaction—small pressure values will lead to largeoutputs.

FIG. 8C shows an illustration of the custom mapping mode where themapping of input to output can be user-defined. One illustrative exampleof this pressure type would be to map 1/3 of the input pressure to alogarithmic curve and the remaining region to a more linear curve asshown in FIG. 7C.

Accelerometer

When using the 3rd generation iPad® and its on-board accelerometer anaverage sampling rate of 30 Hz was achieved with floating point valuesranging from approximately −1.271802 to +1.215422 (for full range). Atypical accelerometer reading in the implementation shown in FIG. 3resulted in a range of 0.0 and 0.2 providing approximately 200,000possible discrete values of accelerometer readings. The combination ofthe amplitude resolution of the accelerometer and sampling rate of 33milliseconds allows the use of accelerometers for the system to be veryaccurate and sensitive. Accordingly, the accelerometer data can be usedto augment and/or control musical applications in real-time.

The augmentation and/or control of musical applications can includecontrolling a MIDI interface or any device that can receive data via MCDwired and wireless I/O. For example, the MCD can be used to output MIDImessages to drive an internal (MCD application) sound synthesizer, forexample. The MIDI messages can be augmented based upon the calculatedpressure value as described above. The augmented MIDI messages can thenbe sent to a remote/external MIDI device. The remote MIDI device canthen respond to the received MIDI control messages. For example, theremote MIDI device can be a sound synthesizer capable of receiving MIDIinput. In this example, the calculated pressure values can be used tocreate pressure sensitive sliders, pan-pots, buttons, and other standardGUI interface objects. In addition, velocity trigger modes can beintegrated with “handle” modes by exploiting the z-axis of theaccelerometer. This allows for a handle and button combinationinteraction set up for one or more GUI objects controlling otherhardware such as lighting systems, video projection, etc.

Augmenting data based upon calculated pressure data is not limited toaugmenting audio parameters. The force-feedback based upon the pressuredata can also be used in video games. For example, an accelerator of acar in a racing game can be pressed by a user. The amount of pressureapplied to the accelerator can be used to control the amount ofacceleration of the car in the racing game. As another example, thecalculated pressure data can be used to control the width of a paintbrush in a paint utility application. Another example, the calculatedpressure can be used to simulate playing an instrument in a video game.Yet another example, the calculated pressure can be applied to provideforce-feedback when pushing a heavy door opposed to a light door.

Elastic Bands

Force-feedback can be provided to a device as described above. Inanother implementation, force-feedback can be used to provide users withtactile expression using elastic band materials that are placed on a MCDsuch that the band stretches over the display of the MCD as shown inFIG. 9. The MCD 902 can be used in various MCD interaction scenariosincluding flipping webpages, software applications for musicalinstrument performance, or gaming environments such as bow-and-arrowgames, golfing games, sling-shot games, or providing force-feedback forgames such as Angry Birds®. In the case of Angry Birds®, there would beno need for software modification or hardware modification to theexisting MCDs.

The elastic band can be positioned such that interaction with band,which is placed over the touch-screen results in tactile force-feedback.The elastic band has an initial position 908. Tactile force-feedback isachieved by combining user interaction with the band and touch-screenGUI objects, such as GUI sliders 904 as shown in FIG. 9. For example,the band can be used to mimic a physical stringed instrument asillustrated in FIG. 9 with GUI sliders. When a user moves the GUI slider904, the elastic band will also move to a position 906 and provide atactile force-feedback to the user.

FIGS. 10A-10C illustrate one or more elastic bands incorporated into acase 904 in accordance with an implementation. In this implementation,the bands can be incorporated into a case 902. For example, an elasticband 908 can retract into the case. In the retracted state, a tip 1006of the elastic band 1008 is visible. Using the tip 1006, the elasticband 1008 can be uncoiled and secured to another portion of the case1002. For example, an elastic band could be secured to the case 1002 viamagnets, notches, etc. In one implementation, the case 1002 can havenotches that secure an elastic band via a node at the end of the elasticband. Multiple elastic bands 1012, 1008, and 1010 can be integrated intothe case 1002.

As shown in FIGS. 10A-10C, the elastic bands 1008 and 1012 are attachedvertically to the MCD with each elastic band mimicking strings. In otherimplementations, the elastic bands can be attached horizontally. Thepluck velocity and the ensuing tactile force-feedback can be rendered byusing standard (visible or invisible) GUI sliders and its displacementreadings as shown in FIG. 8. In these implementations, the userinteracts with the string and the touch-screen where a pluck of thestring is measured by the slider's displacement amount when the userplucks the string. The slider displacement is rendered with a swipeaction on the touch-screen. The band itself acts to provide tactileforce-feedback. The combination of the two provides for tactileforce-feedback interaction with applications running on the MCD or whendriving remote software applications. The use of elastic bands allowsfor placement of single or multiple strings, depending on size of MCD;orientation of the string placement (vertical or horizontal for morestrings but shorter frets, for example). This configuration also allowsfor multiple “frets” depending on number of sliders employed withforce-feedback similar to playing a small guitar. This set up can bebuilt as part of standard MCD protection cases as shown in FIGS.10A-10C.

FIG. 10B depicts the side view when the elastic band is beinguncoiled/extended to “click-on” to the opposite side on of the MCD (thisexample shows the vertical configuration but can also be used inhorizontal band mode). FIG. 10A shows the coiled/retracted state of theelastic bands which allows for normal use of the MCD. FIG. 10C shows thesame design from the top. The elastic bands do not have to be fixed andcan move side-to-side for custom placement as needed by the user.

The GUI sliders can be visible or invisible to the user. In the latter,the background of the display canvas will be visible. Anotherconfiguration example is shown in FIG. 11. Here the sliders are“invisible” and form the background of the display. The invisible sliderin effect allows for the band interaction measurement on a horizontalpixel level, which can allow up to a maximum of 120 invisible horizontalsliders or 720 vertical invisible sliders for modern smartphones.

In another implementation, sliders are bypassed entirely. In thisexample, the entire screen becomes accessible where the starting pointx,y coordinates and ending point x,y coordinates are used to determinepluck intensity. Pluck velocity, for example, could simply be computedby computing:

${v = {\frac{d}{t} = \frac{x_{1} - x_{0}}{t_{1} - t_{0}}}},$

where d is the distance, t is time, x1-x0 is the distance moved, andt1-t0 is duration of the pluck. This is shown in FIG. 12. The change inthe y-direction can also be used. For example, the change in they-direction can be measured and used to produce a slide effect. Forexample, the initially y-position can be used to determine a startingpitch and the ending y-position can be used to determined the endingpitch. The starting pitch and the ending pitch can be used to generate aslide effect.

FIG. 13 shows another configuration in horizontal mode used to flipthrough application pages on a tablet. In this example, a simplesideways “pluck” is shown.

FIG. 14 shows another configuration used for the popular game AngryBirds®. In this example, the band is used to provide force-feedback asone would expect when increasing tension of the sling-shot in the game.The sling shot is pulled from its rest position and released to shootthe “bird” towards the desired target. No software or hardwaremodification is need to the existing software or MCD.

FIG. 16 is an illustration of an implementation utilizing conductivestrings. The device 1602 shown in FIG. 15 includes a touch screen 1607,which is typically not flush with the outermost surface of the device1602. This is particularly true where a case 1606 surrounds the device1602. As most touch screens require physical contact to register a“touch”, the user must depress the string 1608 (for example at 1601)sufficiently to also engage their finger with the screen 1607. However,in one implementation, the string 1608 is conductive or otherwiseinteractive with the touch screen 1607. For example, the string 1608 maybe conductive elastic string, including through the use of a conductivecoating. Thus, in this implementation, the user need not touch thescreen with their finger, but only engage the string 1608 so the string1608 touches the screen 1607.

FIG. 17 is an illustration of an implementation with an alternativestring storage mechanism. The string 1708 is connected to the device1702 (or to a case) at a first location 1712. That connection may be apivotable connection allowing the string to rotate relative to thedevice 1702. The string 1708 can be connected across the screen of thedevice 1702 at play mode location 1713. The play mode location removablysecures the string 1708 across the screen. A rest mode location 1714 ispositioned along the same side of the case or device 1702 as the firstlocation 1712. The rest mode location 1714 removably secures the string1708 so as to not obstruct the screen. The rest mode location 1714 andthe play mode location 1713 may comprise clamps and can be moveableeither in a continuous way or with moveable “grid units” so that it canbe adjusted and customized to different software applications. Multiplestrings can be attached either horizontally and/or vertically.

FIGS. 18A-B are illustrations of an implementation utilizingelectroactive polymers (“EAP”). The strings 1808 comprise one or moreEAP. Using EAP, electrical energy will be produced when plucking thestring 1808. Plucking strength and electrical energy produced will beproportional. A processing mechanism 1810 can translate the electricalenergy into information about the interaction with the string 1808. Forexample, the energy can be amplified and converted to a digital signal,such as by an amplifier 1812 and an analog-to-digital converter 1811.The digital signal can be sent to mobile device for processing, mapping,visualization, etc. This can be done via simple connection to device I/Ointerface or via WiFi/Bluetooth built into microcontroller or the like.The processing mechanism 1810 can be charged or connected to computervia identical I/O format on add-on in case.

FIG. 15 is a block diagram of a computer system in accordance with anillustrative implementation. The computer system or computing device1500 can be used to implement a mobile computing device, cell phones,clients, servers, etc. The computing system 1500 includes a bus 1505 orother communication component for communicating information and aprocessor 1510 or processing circuit coupled to the bus 1505 forprocessing information. The computing system 1500 can also include oneor more processors 1510 or processing circuits coupled to the bus forprocessing information. The computing system 1500 also includes mainmemory 1515, such as a random access memory (RAM) or other dynamicstorage device, coupled to the bus 1505 for storing information, andinstructions to be executed by the processor 1510. Main memory 1515 canalso be used for storing position information, temporary variables, orother intermediate information during execution of instructions by theprocessor 1510. The computing system 1500 may further include a readonly memory (ROM) 1520 or other static storage device coupled to the bus1505 for storing static information and instructions for the processor1510. A storage device 1525, such as a solid state device, magnetic diskor optical disk, is coupled to the bus 1505 for persistently storinginformation and instructions.

The computing system 1500 may be coupled via the bus 1505 to a display1535. An input device 1530, such as a keyboard, may be coupled to thebus 1505 for communicating information and command selections to theprocessor 1510. In another implementation, the input device 1530 has atouch screen display 1535. The input device 1530 can include a cursorcontrol, such as a mouse, a trackball, or cursor direction keys, forcommunicating direction information and command selections to theprocessor 1510 and for controlling cursor movement on the display 1535.

According to various implementations, the processes described herein canbe implemented by the computing system 1500 in response to the processor1510 executing an arrangement of instructions contained in main memory1515. Such instructions can be read into main memory 1515 from anothercomputer-readable medium, such as the storage device 1525. Execution ofthe arrangement of instructions contained in main memory 1515 causes thecomputing system 1500 to perform the illustrative processes describedherein. One or more processors in a multi-processing arrangement mayalso be employed to execute the instructions contained in main memory1515.

Mounting System

In one implementation the mobile computing device (MCD) case 1002 isaugmented in its functionality and utilized as a hands-free, MCDmounting system 2001. The mounting system 2001 illustrated in FIG. 19A-Bis applicable for hanging a MCD from an object such as for situations ascommonly encountered during travel in automobiles. FIGS. 19A and 19Bshow a typical scenario where the case is attached to the support of arearview mirror as commonly found in automobiles as well othersituations.

The mounting system 2001, which allows it to function like aconventional and traditional MCD case that has found great utility inprotecting the MCD 206, and in this implementation, the case 1002further functions as mounting device 2001 for, but not limited to,automobiles, freeing the hands for full focus on driving without theneed for additional hardware such as windshield/dashboard MCD mountinghardware. Not only does the mounting system 2001 allow for hands-freenavigation, it also strategically positions the MCD 206 in an area wheredrivers constantly and commonly seek information—traffic behind theautomobile. As such, the positioning of the MCD 206 is not only in afamiliar location, but in a location where the driver's vision isexpected to focus on during situations such as passing vehicles, makingturns, or backing out of garages, for example. Helpfully, this positionis also typically viewable by passengers in the automobile. The mountingsystem 2001 greatly helps in keeping the driver's eyes in front, aroundthe windshield, and around the rearview mirror location, thus,contributing to safety while driving and utilizing navigation softwarethat is commonly installed in MCDs, for example. As mounting system 2001is part of the case 1001 with no external additional hardware needed tobe carried or installed, the system 2001 can be set up and removed fromsuch places as rearview mirror systems easily and safely.

A mounting system 2001 includes mounting frame 2020 having a verticalsupport 2021, preferably but not limited to, two parallel verticalsupports 2021 a, 2021 b, engaged with a horizontal handle 2022 at oradjacent a first end of the vertical supports 2021. The vertical support2021 is stowable within the case 1002. In one embodiment, the verticalsupport 2021 is slidable to be disposed entirely or substantially withinthe case 1002 and extendable from the case 1002, such as extendable alength substantially equal to the length of the case 1002. For example,the vertical support 2021 may form a “u” shape in conjunction with thehorizontal handle 2022. In one embodiment, the horizontal handle 2022 isperpendicular to the vertical support 2021. The horizontal handle 2022is hinged to allow at least a portion of the horizontal handle 2021 topivot relative to the vertical support 2021. It should be appreciatedthat in alternative embodiments, the movement may be one or more ofpivoting, bending, flexing, and rotating. The horizontal handle 2022thus provides an open state (e.g., FIG. 20C and FIG. 22A) and a closedstate (e.g. FIG. 20B) (as well as transitions between those states). Inthe open state, a support 22, such as the post of a rearview mirror 21,can pass between the horizontal handle 2022 and the vertical supports2021. In the closed state, the support 22 is encompassed by the verticalsupports 2021, the horizontal handle 2022, and the case 1002 having theMCD 206.

The procedure for setting up mounting system 2001 for rearview mirrors21 is shown in FIG. 20A—FIG. 20E. In FIG. 20A, mounting system 2001 isin its initial state where it looks, feels, and functions similar totraditional MCD cases and acts as to protect the handheld device. FIG.20B show the mounting system 2001 holding mechanism pulled out of themounting system 2001 case—the amount of retraction of theholding/support system is adjustable and can be approximately as long asthe height of the MCD 206. As shown in FIG. 20C, the MCD 206, having themounting system 2001 in an extended state, is pushed against therearview mirror support 22. Specifically, the horizontal handle 2022 ispressed against the support 22. This causes the pivoting portion of thehorizontal handle 2022 to pivot, creating an opening, which allows it tobe secured to the rearview mirror support 22 by “capturing” the support22 within the frame 2020. The pivoting portion 2024 is biased to aclosed state and once the support 22 passes through the created opening,the pivoting portion 2024 pivots back to a closed state securing the MCD206 to the rearview mirror 21. The height of the mounting system 2001can then be adjusted to the user's needs as further described below:flush against the bottom of the rearview mirror or hanging further awayas needed. In one embodiment, one or more of the horizontal handle 2022,the pivoting portion 2024, and all or a portion of the vertical supports2021 have a high-friction surface, such as a rubbery materials orcoating, to utilize frictional forces to further secure and stabilizethe position of the MCD 206. In another implementation one or more ofthe horizontal handle 2022, the pivoting portion 2024, and all or aportion of the vertical supports 2021 present a flat surface forengaging the support 22, for example rather than rounded, to allowmaximal contact area—and thus friction and in turn stability—between theMCD 206 and support 22, additionally helping to movement issues.

In another implementation, shown in FIG. 21A, the horizontal handle 2022includes an indent 2023, notch or the like configured to receive thesupport 22, such as the support portion of a rear view mirror 21. Theindent 2023 may be position entirely on the pivoting portion 2024 or mayonly include portion thereof on the pivoting portion 2024. The indent2023 contributes to minimizing movement and securing the MCD 206 to thesupport 22, such as the rearview mirror support 22 of FIG. 21A. Inanother implementation, shown in FIG. 21B, to additionally make themounted MCD 206 prone to less movement, a lower horizontal fasteninghandle 2025 can be below the horizontal handle 2022, preferablyconfigured to be a distance slightly greater than the thickness of thesupport 22. This allows for added stability as the upper horizontalhandle 2022 and the lower horizontal handle 2025, particularly where theupper horizontal handle 2022 has an indent 2023 and the lower horizontalhandle 2025 has a corresponding indent 2026, will secure, effectivelyclamping around the support 22. The horizontal handles 2022, 2025 can becovered and/or be created with high friction material such as rubber tomaximize stability. In one alternative, one or more of the horizontalhandle 2022 and the lower horizontal handle 2025 comprise flexibleand/or elastic material, allowing for the respective handle to deformaround the support 22 to accommodate it and provide for a more securemounting.

The entire set up procedure: (1) retracing the mounting system 2001, (2)securely snapping the mounting system 2001 onto the support 22 (pushingmotion), and (3) optionally adjusting the length as necessary is quick,easy, and extremely intuitive.

In a further embodiment, the mounting system 2001 handle is at rest andhorizontal to the MCD 206 and can be bent towards and away from the MCD206. This design allow for quick, easy, and uncomplicated release of theMCD 206: (1) pulling of the MCD 206 to engage the support 22 with thepivot portion 2024, pivoting the pivot portion 2024 away from the MCD206 (FIG. 22A), allowing removal of the support 22 from the frame 2020and (2) pushing mounting system 2001 back into the case 1002 which willresult in transforming the MCD case 1002 back to its normal state asshown in FIG. 22B (similar to the starting state of FIG. 20A). Inanother implementation, the pivot portion 2024 of the horizontal handle2022 pivots different amounts: (1) pivoting towards MCD 206 is lessresistive and (2) pivoting away from the MCD 206 is more resistive whichallows for extra safety as releasing the MCD will require a little moreforce when pulling. This allows for scenarios where heavier MCDs'inadvertent release is minimized. The bias exhibited by the pivotingportion against movement from the resting state may be accomplished bymechanisms known in the art, such as use of a living hinge, materialproperties, or external bias forces such as springs or elasticmaterials.

FIG. 23 shows an illustrative example of the structure of the mountingsystem 2001 by internal view of the case 1002. FIG. 24 shows asimplified view of the embodiment of FIG. 23. The mounting system 2001frame 2020 can be pulled/retracted to approximately the size of the case1002 providing flexibility in adjusting the MCD 206 to the user's needsincluding view height and positioning. To limit how much the frame 2020can be extended from the case 1002—preventing from detaching itself fromthe case—a blocking module 2040 is attached at the top part as shown inFIG. 24B. In one embodiment, the blocking module 2040 is a flange orstructure within the case 1002 substantially in the same plane as theframe 2020. The frame 2020, in the illustrated embodiments, includes ablocking horizontal arm 2032 that engages with the blocking module 2040to limit the amount the frame 2020 can be extended from the case 1002.In one embodiment, the arm 2032 is a plurality of extending portionsrather than a solid extending arm of FIG. 24A.

In a further embodiment, also show in FIG. 23, a locking mechanism 2050is provided. The locking mechanism 2050 allows the frame 2020 to lock atvarious amounts of extension from the case 1002, such as via aninterlocking system 2059 example is shown in FIG. 23 and FIG. 25A. Inthe illustrated embodiment, the interlocking system comprises aplurality of opposing teeth or protrusions 2053, 2033. In oneembodiment, a rail 2051 is provided that is secured to the case 1002 orthe MCD 206. The frame 2020 is slidable relative to the rail 2051. Therail 2051 may have one or more vertical rail elements 2051 a, 2051 bcorresponding to the respective vertical supports 2021 a, 2021 b. Eachvertical support 2021 engages and is slidable along a corresponding rail2051 in the embodiment of FIG. 23. The interlocking system 2059 providesfor engagement between the rail 2051 and the vertical support 2021. Itshould be appreciated that the vertical support 2021 can be shaped toprovide varied resistance to address the different weights of MCDs 206.This includes, but is not limited to, shapes that are square,rectangular, triangular, round, etc. FIG. 25A shows a roundedimplementation example.

As discussed above, the vertical supports 2021 may engage with avertical rail element 2051. The engagement may provide one or more ofthe locking mechanism 2050 function and securing the vertical supports2021 to the case 1002 and/or MCD 206. An example implementation of avertical rail element 2051 is shown in FIG. 25B with the correspondingportion of the interlocking mechanism 2050 being rounded teeth 2053pointing towards the left hand side of the image and a track 2054.Another view of the vertical rail element 2051 is shown in FIG. 25C,where the teeth 2053 point away from the image page and the verticalrail element 2051 includes a retention flange that keeps the verticalsupport 2021 a in place and aligned.

In one embodiment, the locking mechanism 2050 is biased to a lockedstate (FIG. 26A) and may be moved to a unlocked state (FIG. 26B) withenough force, the interlocking mechanism 2059 can be separated bypulling or pushing the dual handle system, as shown in FIGS. 26A and26B. That implementation illustrates an embodiment where the verticalhandle supports 2021 are bendable or flexible away from the verticalrail element 2051. Specifically, one or more of the vertical supports2021 are bendable away, such as towards the center of the mounting frame2020 in plane of the frame 2020, to disengage the teeth 2033 from theteeth 2053 of the rail 2051. This allows the vertical support to slidealong the rail 2051 free of the interlocking mechanism 2059 forintuitive user interaction design that allows for smooth pulling orpushing gestures. This can be accomplished by a squeezing action on theportion of the vertical supports 2021 extending beyond the case 1002.FIG. 26A shows the vertical support 2021 a in rest and upright, unbentstate where it is interlocked with the fixed interlocking pair 2033,2053. In FIG. 26B, the vertical support 2021 is bent, such as bysqueezing in conjunction with the opposite vertical support 2021 b (notshown in FIG. 26B) causing a gap to occur between the vertical support2021 a and the rail 2051 a, specifically disengaging the respectivecomponents 2033, 2053 of the interlocking mechanism 2059. This allowsfor smooth pulling/pushing until the force exerted by the user, such asthe squeezing pressure, is released by the user. Once pressure isreleased, the interlocking mechanism snaps back (such as biased by theoriginal configuration and elasticity of the material) the handle backsecurely into its rest in place. In FIG. 26B, amount of bending isoptionally limited by a blocking module 2040, preventing overbendingthat may damage the vertical support 2021 a.

FIG. 27 shows another implementation of the top horizontal handlepivoting portion 2024 of mounting system 2001. Here, one of the verticalsupports 2021 includes a horizontal protrusion that engages with andslightly overlaps with the horizontal handle 2022. The horizontal handle2022 can be in part or fully, more (or less or equally) flexible andbendable compared to the vertical supports 2021. This mechanism allowsfor additional safety in securing heavier MCDs 206 to the rearviewmirror support as the clip will not as easily open, preventingunintended release of the MCD from the mirror. This mechanism does notimpede on releasing the MCD 206 when pulling it with reasonable force,thus allowing for extra safety when hung on to the mirror 21, forexample.

In another implementation the top handle is safely closed with a lockingmechanism for added safety. This mechanism, however, requires additionalmanual release action of the horizontal locking mechanism.

In another implementation, the dual mounting system 2001 is rendered viafilaments, including elastic filaments, as in the string model wherebyit can be pulled out from the MCD case at one of the corner toppositions and secured around the rearview mirror support and clipped onthe other side of the top of the MCD case. Once finished, the clippedend can be unclipped and retracted back to the case.

In another implementation, the mounting system 2001 is rendered viasingle or multiple filaments 2070, including elastic filament types.Thus, in one embodiment shown in FIGS. 28A-C, a filament 2070 isprovided attached to the case 1002. The filament 2070 is detachable atone end so as to allow the filament 2070 to loop over a support 22 andis then reattachable to the case at a reattachment point 2075. Thereattachment point 2075 may be the same or different as the location thefilament is secured to the case 1002 when stowed. FIG. 28A shows afilament 2070 aligned vertically for landscape mounting of the MCD 206.FIG. 28B shows a filament 2070 aligned horizontally for portraitmounting of the MCD 206. The filament 2070 may be elastic or inelastic.In one embodiment, the filament 2070 comprises a high-friction surfacefor aiding in securing the MCD 206 to the support 22. Thisimplementation bypasses the need for a separate retracting mechanism.

In another implementation visuals on the touchscreen/monitor of the MCDis variably transparent allowing the user to see through the MCD. Inthis implementation the “wallpaper” background of the OS is not a staticpicture but real-time video captured by the MCDs camera. This designcreates a quasi-cloaking feature, which can further add to safety whendriving. For example, when using the MCD as a navigation system with amobile mapping software such as Google Maps, the map itself can berendered semitransparent allowing map information and real-time roadactivity information.

In another implementation, mounting system 2001 is a standalone add-onfor existing cases. In these instances mounting system 2001 can beattached to conventional cases with permanent or semi-permanentadhesives or other means of securing it to existing MCD cases.

In another implementation, mounting system 2001 includes a positionableframe 2020, such that the vertical supports 2021 can be bent to retain adesired shape such as bent to form an angled frame for to customize thefit when mounted to a support 22.

In another implementation mounting system 2001 can engage with a MCD 206and/or a case 1002 in a “landscape” orientation. In one embodiment, thevertical supports 2021 are flexible and elastic, allowing the supportsto curve within the landscape orientated case 1002 such that thevertical support 2021 may be longer than the height of the landscapeoriented case 1002.

In another implementation mounting system 2001 the vertical support 2021is implemented via telescoping technologies, such as akin to travelluggage systems, where the frame can be can be pulled and retractedthrough the telescoping mechanisms.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular implementations of particularinventions. Certain features described in this specification in thecontext of separate implementations can also be implemented incombination in a single implementation. Conversely, various featuresdescribed in the context of a single implementation can also beimplemented in multiple implementations separately or in any suitablesubcombination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asubcombination or variation of a subcombination. Moreover, theseparation of various system components in the implementations describedabove should not be understood as requiring such separation in allimplementations, and it should be understood that the described programcomponents and systems can generally be integrated in a single softwareproduct or packaged into multiple software products.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. Thus, particular implementations of the subject matter havebeen described. Other implementations are within the scope of thefollowing claims.

What is claimed is:
 1. A case for holding a mobile computing devicecomprising: a housing configured to receive the mobile computing device;an elastic string affixed at a first portion to the case at a firstlocation and having a tactile connection with a screen of the mobilecomputing device; and a play mode location on the case for removablyreceiving a second portion of the string; wherein the string extendsover the mobile computing device when affixed to the first location andremovably received at the play mode location, and wherein the elasticstring has an initial displacement and a pluck displacement, the initialdisplacement being the elastic string at rest, the pluck displacementbeing the displacement of the elastic string in response to a pluckdisplacement by a user, and wherein the elastic string is released fromthe pluck displacement and returns to the initial displacement at apluck velocity, the pluck velocity dependent on the tactile connection,initial displacement, and pluck displacement, and wherein data withinthe mobile computing device is augmented based upon the pluck velocity.2. The case of claim 1, further comprising: a notch configured toreceive a node; and a retractable elastic band including the node. 3.The case of claim 1, wherein the string is electroactive polymer.
 4. Thecase of claim 1, wherein the string is conductive.
 5. The case of claim1, further comprising resistive material affixed to the house configuredto compress when pressure is applied to the computing device, whereinpressure can be measured by the computing device based upon thecompressed resistive material, and wherein data within the mobilecomputing device is augmented based upon the pressure.
 6. The case ofclaim 1, further comprising a rest mode location for removably receivingthe string wherein the wherein the string does not cover the mobilecomputing device when affixed to the first location and removablyreceived at the play mode location.
 7. The case of claim 1, wherein thepluck velocity is determined by an equation of$v = {\frac{d}{t} = \frac{x_{1} - x_{0}}{t_{1} - t_{0}}}$ wherein “v” isthe pluck velocity, “d” is a distance, “t” is a time, “x1-x0” is adistance displaced, and t1-t0″ is a duration of the pluck displacement.